Method for recommending continuing education to health professionals based on patient outcomes

ABSTRACT

Methods and systems for providing health professionals with continued education are based on performance gaps identified from patient data available in transactional systems of record. The methods can include creating a repository of educational material, measuring patient and team level performance gaps, associating the identified performance gaps with appropriate educational material, alerting the person about the appropriate educational material, capturing a user&#39;s interaction with the educational materials, and issuing credits or rewards for substantial consumption of the educational materials.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

This invention was made with government support under Small BusinessInnovation Research Program awarded by the National Science Foundation.The government has certain rights in the invention.

BACKGROUND OF THE INVENTION 1. Field of the Invention

Embodiments of the invention relates generally to education methods.More particularly, the invention relates to methods for providingcontinuing education and nudges to health professionals, where theeducation modules may be based on patient outcomes.

2. Description of Prior Art and Related Information

The following background information may present examples of specificaspects of the prior art (e.g., without limitation, approaches, facts,or common wisdom) that, while expected to be helpful to further educatethe reader as to additional aspects of the prior art, is not to beconstrued as limiting the present invention, or any embodiments thereof,to anything stated or implied therein or inferred thereupon.

All health professionals are required to take continued educationcredits throughout their career in order to maintain their licenses andcredentials current. This process was originally designed to be separatefrom the workplace setting. However, the speed of innovation in healthcare and competitive pressures on health care institutions require thattraining and education for health professionals be aligned to achieveresults in patient outcomes. Learning management systems are the currenttechnological solution used to deploy training and educational contentto health professionals in workplace settings. However, these softwaresystems have been unable to establish the relationships between a pieceof training (or educational content/nudge) with any improvements inclinical outcomes.

Until now, the computer functionality to draw data from a database withpatient information (like that found in Electronic Health Records andpatient reported outcomes) and connect it with learning managementsystems to enable computations that train machine learning models toadjust individual patient health risk factor scores, assign patients toworkflows and recommend personalized educational content was previouslyunavailable.

In view of the foregoing, there is a need for education methods thataddress the shortcomings of conventional methods.

SUMMARY OF THE INVENTION

Embodiments of the present invention are drawn to an automatic mechanismfor providing health care professionals with personalized continuededucation content and nudges which address the skills necessary toimprove the clinical outcomes of their specific patient populations.

This learning management system can include a computational enginecapable of the following features. (1) Generating patient health riskfactor scores or mathematical representations of individual levelclinical and non-clinical factors through a machine learning model topredict potential adverse health episodes; (2) Assigning patientsaccording to their health risk factor score to workflows using machinelearning models that are trained to improve assignment logic and rulesaccording to their effectiveness for achieving patient outcomes; (3)Storing and coding libraries of continued educational content by patienthealth risks, workflow components and learner preferences; (4)Identifying individual learner interests, learning styles andpreferences by recording and sorting through their experiences engagingwith the learning management system; (5) Recommending individualizedcontinued education content to health professionals that addressesskills related to the health risks of their specific patient population,compliance with recommended workflows, learning styles and preferences;(6) Training machine learning models to identify which continuededucation content is correlated with improved patient outcomes and underwhat conditions; and (7) Presenting, capturing and reporting informationrelated to the consumption of educational content which can be used togrant professionals with continued education credits.

The primary objective of SMART Patient-Centered Medical Home Manager(PCMH Manager) is to reduce and manage chronic disease by enablingteam-based care in primary care practices. Embodiments of the presentinvention propose to apply precision-education instructional theoryaround team competencies to promote situational awareness, enhancedcommunication, defined role clarity, improved coordination andleadership support to improve patient outcomes. Various methods,according to embodiments of the present invention, include a web-basedsoftware that collects clinical and socioeconomic patient-level data anduse machine learning to establish patient-specific risk scores topredict adverse health episodes. The software evaluates theeffectiveness of team-level workflows and provides health professionalswith individualized treatment plans and continued education to enhancepatient outcomes.

According to one method of the present invention, the health careknowledge is suggested based on the performance health care provider orthe care team(s) to which the health care provider is assigned. Themethod provides for creating or loading granules of health careknowledge and associating continuing education credits or other rewardswith consuming the health care knowledge. Health care knowledgesuggestions can be triggered for the health care professional based onmeasures calculated from patient demographic, clinical, and non-clinicaldata extracted from systems used by the health care provider and thecare team(s) to which the health care provider is assigned. The measurescalculated from patient demographic, clinical, and non-clinical data arecompared to threshold values to decide whether to trigger a suggestionfor a particular health care knowledge granule that are associated withinformation, skills, and aptitudes that are intended to improve thehealth care provider's performance. Therefore, the granules can becreated so that they can be easily consumed in a short time period andhave the potential for impacting patient care.

Methods of the present invention can be applied in a variety ofcontexts. The thresholds to which the measures calculated from patientdemographic and clinical data can be set by a provider organizationbased on established care standards. If the provider is organization isseeking to increase performance level of a particular health careprovider and the care team(s) to which the health care provider isassigned, it can choose to select thresholds that are based onstatistical comparisons against the performance of other care teams inits facilities. This can be extended so that provider organizations canchoose to select thresholds that are based on statistical comparisonsagainst the performance of all care teams that are using a similarcomputer-assisted method.

In one method of the present invention, the health care knowledgeassociated with the triggered suggestion can be delivered immediatelywhen the suggestion is generated so that the health care providerconsumes it at that time. In the alternative, the alerts resulting fromcontent suggestion triggers can be queued up so that the health careprovider can respond to them whenever he or she has the opportunity todecide when to consume the suggested content. The information on how thehealth care professionals responds to the alerts in terms of timing, thecontent selected by the health care provider when offered similaroptions, and other characteristics of the process of responding to thealerts can be captured, synthesized, stored and later used to makepredictions of how the alerts and the content can be delivered tomaximize engagement and effectiveness. This would be a case ofdelivering health care content based on the health care provider'simplied preferences rather than the specific preferences the health careprovider may have established in his or her user profile.

Embodiment of the present invention can use data science techniques toidentify the content consumption patterns of health care providers andcare team(s) whose clinical performance has improved, specificallywhenever causality between the content consumption and the clinicalperformance can be established and incorporate the associated contentconsumption model into the content suggestion trigger determination. Asmore data is available related to the use of this computer-assistedmethod, the manner in which the content suggestions are triggered can betransitioned from just comparison against clinical measure thresholds toa method that incorporates successful content consumption patterns, aswell as the clinical measures, associated with the health care providerand the care team(s) to which the health care provider is assigned.

Embodiments of the present invention provide a method for providingcontinuing education a health care professional comprising continuallyassessing actual clinical performance of the health care professional asrecorded in a clinical data system; creating a set of granules of healthcare knowledge or similar content in a database; associating continuingeducation credits with each granule of health care knowledge of the setof granules of health care knowledge; defining a set of configurableconditions that use data from the clinical data system to triggerappropriate granule suggestions to the health care provider;electronically delivering the appropriate health care knowledge granulebased on the established preferences of the health care provider;recording the health care provider's interaction with the health careknowledge granule; determining whether the interaction with the healthcare knowledge granule is indicative that the health care providerconsumed the health care knowledge granule; and recording continuingeducation credits based on the delivered health care knowledge granuleif there is indication that the health care provider consumed the healthcare knowledge granule.

Embodiments of the present invention further provide a computer-assistedmethod for delivering health care knowledge to a health care provider,based on actual clinical performance recorded in a clinical data systemcomprising participating in interactions with patients and recordingassociated data in the clinical data system; acquiring relevant patientclinical data from the clinical data system; triggering one or moresuggested health care knowledge granules based on performance of thehealth care provider; obtaining a request from the health care providerto deliver the suggested health care knowledge granules; delivering thesuggested granules that were requested by the health care provider;recording an interaction of the health care provider with the healthcare knowledge granule; determining whether the interaction between thehealth care provider and health care knowledge granule was substantive;and recording continuing education credits based on the delivered healthcare knowledge granule if there is indication that the health careprovider had a substantive interaction with the health care knowledgegranule.

These and other features, aspects and advantages of the presentinvention will become better understood with reference to the followingdrawings, description and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the present invention are illustrated as an exampleand are not limited by the figures of the accompanying drawings, inwhich like references may indicate similar elements.

FIG. 1 illustrates a diagram showing one embodiment of the methodologyof the present invention;

FIG. 2 illustrates a block diagram showing one implementation of asystem according to the present invention in a use case associated withthe use of an organization's internal network to access the system;

FIG. 3 illustrates a block diagram showing one implementation of asystem according to the present invention in a use case associated withthe use of a user's Internet Service Provider (ISP) to access thesystem;

FIG. 4 provides a point of care work context diagram according to oneembodiment of the present invention;

FIG. 5 provides a work context diagram for the computational engine,which can absorb data associated with patient risks and feed that to thepatient risk factor machine learning system according to an embodimentof the present invention;

FIG. 6 provides additional details on the functionality of the patientrisk factor machine learning system according to an embodiment of thepresent invention;

FIG. 7 provides additional details on the functionality of a patientworkflow assignment system according to an embodiment of the presentinvention; and

FIG. 8 provides a diagram of the continuous education deployment systemaccording to an embodiment of the present invention.

Unless otherwise indicated illustrations in the figures are notnecessarily drawn to scale.

The invention and its various embodiments can now be better understoodby turning to the following detailed description wherein illustratedembodiments are described. It is to be expressly understood that theillustrated embodiments are set forth as examples and not by way oflimitations on the invention as ultimately defined in the claims.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS AND BEST MODE OFINVENTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the term “and/or” includes any and all combinations of oneor more of the associated listed items. As used herein, the singularforms “a,” “an,” and “the” are intended to include the plural forms aswell as the singular forms, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, steps, operations, elements, components, and/or groupsthereof.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by onehaving ordinary skill in the art to which this invention belongs. Itwill be further understood that terms, such as those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and thepresent disclosure and will not be interpreted in an idealized or overlyformal sense unless expressly so defined herein.

In describing the invention, it will be understood that a number oftechniques and steps are disclosed. Each of these has individual benefitand each can also be used in conjunction with one or more, or in somecases all, of the other disclosed techniques. Accordingly, for the sakeof clarity, this description will refrain from repeating every possiblecombination of the individual steps in an unnecessary fashion.Nevertheless, the specification and claims should be read with theunderstanding that such combinations are entirely within the scope ofthe invention and the claims.

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be evident, however, toone skilled in the art that the present invention may be practicedwithout these specific details.

The present disclosure is to be considered as an exemplification of theinvention and is not intended to limit the invention to the specificembodiments illustrated by the figures or description below.

As is well known to those skilled in the art, many carefulconsiderations and compromises typically must be made when designing forthe optimal configuration of a commercial implementation of any system,and in particular, the embodiments of the present invention. Acommercial implementation in accordance with the spirit and teachings ofthe present invention may be configured according to the needs of theparticular application, whereby any aspect(s), feature(s), function(s),result(s), component(s), approach(es), or step(s) of the teachingsrelated to any described embodiment of the present invention may besuitably omitted, included, adapted, mixed and matched, or improvedand/or optimized by those skilled in the art, using their average skillsand known techniques, to achieve the desired implementation thataddresses the needs of the particular application.

Devices or system modules that are in at least general communicationwith each other need not be in continuous communication with each other,unless expressly specified otherwise. In addition, devices or systemmodules that are in at least general communication with each other maycommunicate directly or indirectly through one or more intermediaries.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Onthe contrary, a variety of optional components are described toillustrate the wide variety of possible embodiments of the presentinvention.

A “computer” or “computing device” may refer to one or more apparatusand/or one or more systems that are capable of accepting a structuredinput, processing the structured input according to prescribed rules,and producing results of the processing as output. Examples of acomputer or computing device may include: a computer; a stationaryand/or portable computer; a computer having a single processor, multipleprocessors, or multi-core processors, which may operate in paralleland/or not in parallel; a general purpose computer; a supercomputer; amainframe; a super mini-computer; a mini-computer; a workstation; amicro-computer; a server; a client; an interactive television; a webappliance; a telecommunications device with internet access; a hybridcombination of a computer and an interactive television; a portablecomputer; a tablet personal computer (PC); a personal digital assistant(PDA); a portable telephone; application-specific hardware to emulate acomputer and/or software, such as, for example, a digital signalprocessor (DSP), a field programmable gate array (FPGA), an applicationspecific integrated circuit (ASIC), an application specificinstruction-set processor (ASIP), a chip, chips, a system on a chip, ora chip set; a data acquisition device; an optical computer; a quantumcomputer; a biological computer; and generally, an apparatus that mayaccept data, process data according to one or more stored softwareprograms, generate results, and typically include input, output,storage, arithmetic, logic, and control units.

“Software” or “application” may refer to prescribed rules to operate acomputer. Examples of software or applications may include: codesegments in one or more computer-readable languages; graphical andor/textual instructions; applets; pre-compiled code; interpreted code;compiled code; and computer programs.

The example embodiments described herein can be implemented in anoperating environment comprising computer-executable instructions (e.g.,software) installed on a computer, in hardware, or in a combination ofsoftware and hardware. The computer-executable instructions can bewritten in a computer programming language or can be embodied infirmware logic. If written in a programming language conforming to arecognized standard, such instructions can be executed on a variety ofhardware platforms and for interfaces to a variety of operating systems.Although not limited thereto, computer software program code forcarrying out operations for aspects of the present invention can bewritten in any combination of one or more suitable programminglanguages, including an object oriented programming languages and/orconventional procedural programming languages, and/or programminglanguages such as, for example, Hypertext Markup Language (HTML),Dynamic HTML, Extensible Markup Language (XML), Extensible StylesheetLanguage (XSL), Document Style Semantics and Specification Language(DSSSL), Cascading Style Sheets (CSS), Synchronized MultimediaIntegration Language (SMIL), Wireless Markup Language (WML), Java.™.,Jini.™., C, C++, Smalltalk, Python, Perl, UNIX Shell, Visual Basic orVisual Basic Script, Virtual Reality Markup Language (VRML),ColdFusion.™. or other compilers, assemblers, interpreters or othercomputer languages or platforms.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). The program code may also be distributed among a plurality ofcomputational units wherein each unit processes a portion of the totalcomputation.

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

Further, although process steps, method steps, algorithms or the likemay be described in a sequential order, such processes, methods andalgorithms may be configured to work in alternate orders. In otherwords, any sequence or order of steps that may be described does notnecessarily indicate a requirement that the steps be performed in thatorder. The steps of processes described herein may be performed in anyorder practical. Further, some steps may be performed simultaneously.

It will be readily apparent that the various methods and algorithmsdescribed herein may be implemented by, e.g., appropriately programmedgeneral purpose computers and computing devices. Typically, a processor(e.g., a microprocessor) will receive instructions from a memory or likedevice, and execute those instructions, thereby performing a processdefined by those instructions. Further, programs that implement suchmethods and algorithms may be stored and transmitted using a variety ofknown media.

When a single device or article is described herein, it will be readilyapparent that more than one device/article (whether or not theycooperate) may be used in place of a single device/article. Similarly,where more than one device or article is described herein (whether ornot they cooperate), it will be readily apparent that a singledevice/article may be used in place of the more than one device orarticle.

The term “computer-readable medium” as used herein refers to any mediumthat participates in providing data (e.g., instructions) which may beread by a computer, a processor or a like device. Such a medium may takemany forms, including but not limited to, non-volatile media, volatilemedia, and transmission media. Non-volatile media include, for example,optical or magnetic disks and other persistent memory. Volatile mediainclude dynamic random access memory (DRAM), which typically constitutesthe main memory. Transmission media include coaxial cables, copper wireand fiber optics, including the wires that comprise a system bus coupledto the processor. Transmission media may include or convey acousticwaves, light waves and electromagnetic emissions, such as thosegenerated during radio frequency (RF) and infrared (IR) datacommunications. Common forms of computer-readable media include, forexample, a floppy disk, a flexible disk, hard disk, magnetic tape, anyother magnetic medium, a CD-ROM, DVD, any other optical medium, punchcards, paper tape, any other physical medium with patterns of holes, aRAM, a PROM, an EPROM, a FLASHEEPROM, any other memory chip orcartridge, a carrier wave as described hereinafter, or any other mediumfrom which a computer can read.

Various forms of computer readable media may be involved in carryingsequences of instructions to a processor. For example, sequences ofinstruction (i) may be delivered from RAM to a processor, (ii) may becarried over a wireless transmission medium, and/or (iii) may beformatted according to numerous formats, standards or protocols, such asBluetooth, TDMA, CDMA, 3G and the like.

Where databases are described, it will be understood by one of ordinaryskill in the art that (i) alternative database structures to thosedescribed may be readily employed, (ii) other memory structures besidesdatabases may be readily employed. Any schematic illustrations andaccompanying descriptions of any sample databases presented herein areexemplary arrangements for stored representations of information. Anynumber of other arrangements may be employed besides those suggested bythe tables shown. Similarly, any illustrated entries of the databasesrepresent exemplary information only; those skilled in the art willunderstand that the number and content of the entries can be differentfrom those illustrated herein. Further, despite any depiction of thedatabases as tables, an object-based model could be used to store andmanipulate the data types of the present invention and likewise, objectmethods or behaviors can be used to implement the processes of thepresent invention.

Unless specifically stated otherwise, and as may be apparent from thefollowing description and claims, it should be appreciated thatthroughout the specification descriptions utilizing terms such as“processing,” “computing,” “calculating,” “determining,” or the like,refer to the action and/or processes of a computer or computing system,or similar electronic computing device, that manipulate and/or transformdata represented as physical, such as electronic, quantities within thecomputing system's registers and/or memories into other data similarlyrepresented as physical quantities within the computing system'smemories, registers or other such information storage, transmission ordisplay devices.

In a similar manner, the term “processor” may refer to any device orportion of a device that processes electronic data from registers and/ormemory to transform that electronic data into other electronic data thatmay be stored in registers and/or memory or may be communicated to anexternal device so as to cause physical changes or actuation of theexternal device.

Broadly, embodiments of the present invention provide methods andsystems for providing health professionals with continued educationbased on performance gaps identified from patient data available intransactional systems of record. The invention can include creating arepository of educational material, measuring patient and team levelperformance gaps, associating the identified performance gaps withappropriate educational material, alerting the person about theappropriate educational material, capturing a user's interaction withthe educational materials, and issuing credits or rewards forsubstantial consumption of the educational materials.

FIG. 1 illustrates the methodology of a system 10 according to anembodiment of the present invention. A point of care 12 is shown. Thepoint of care 12 is a point at any time where the health care provider14 meets the patient for a consultation or other interaction occursbetween the health care provider 14 and a patient that impacts thepatient health record in an electronic health record (EHR) or otherclinical data systems 16. The health care provider 14 can utilize anumber of devices 18 at the point of care 12 to connect to EHRs andother clinical data systems 16 to access patient data that can supportthe consultation or interaction. In addition, the health care provider14 may enter information regarding the current consultation orinteraction into the EHR of other clinical data systems 16. This canoccur through a desktop or laptop computer 20, a tablet or similarcomputing device 22, or through a cellular phone 24 that is connected tothe health care provider organization's internal network.

The information in the EHRs and other clinical data systems 16 may beuploaded by the patient information database 26 of the system through avariety of technical processes. The patient information database 26 cananalyze the performance of the healthcare provider 14 or the careteam(s) to which he or she is assigned to determine whether it fallswithin defined thresholds. If it does not, the patient informationdatabase 24 can display a message or send an alert to the healthcareprovider 14 requesting that the healthcare provider read, view, orconsume one or more specifically suggested health care knowledgegranules that are available in the learning management computationalengine 28.

The specified health care knowledge granules may relate directly to thethresholds that the healthcare provider and the care team(s) to which heor she is assigned did not meet. The relationship between the healthcare knowledge granules and the specific thresholds has been previouslyestablished by the system administrator or other user to whom suchprivileges have been assigned.

The health care provider 14 may receive the alert through pre-definedcommunications channels, which can include the system 30 or a messagethrough other means, such as an SMS message. The healthcare provider 14may elect to read, view, or consume the suggested health care knowledgegranules immediately or wait until he or she deems it appropriate to doso. At such a time, the healthcare provider 14 may access the specifichealth care knowledge granules in the learning management computationalengine 28. The interaction between the health care provider 14 and eachhealthcare knowledge granule may be captured by learning managementcomputational engine 28, including whether the content was played in itsentirety in the case of multi-media granules, the amount of time thehealth care provider interacted with the content, and any otherparameter deemed to be relevant to determine whether the healthcareprovider 28 substantially consumed the healthcare knowledge granule.

The learning management computational engine 28 can evaluate thecaptured parameters related to the interaction between the healthcareprovider 28 and the granule against pre-specified thresholds. If theparameters captured as a result of the health care provider 14interactions meet the specific criteria associated with the particularhealthcare knowledge granule, as previously established by the system'sadministrator, the continuing education credits, recognition or otherrewards associated with the healthcare knowledge granule can be issuedin favor of the health care provider 14. The information regardinghealthcare provider 14 continuing education credits or rewards may bestored in the patient information database 26 under the profileassociated with the health care provider 14 and can be used as an inputtowards future decisions as to whether suggest the same healthcareknowledge granule or different ones whenever the performance of thehealthcare provider 14 and the care team(s) to which he or she isassigned is once again analyzed or reviewed.

The method to select the content that is to be presented to the healthcare provider 14 can be implemented in a number of ways that take intoconsideration the performance of the health care provider 14 based onthe information in the EHRs and other clinical data systems 16. Theclinical standards and thresholds against which performance is evaluatedmay be defined and configured in the patient information database 26.

The present invention is not limited to any particular network topologyor deployment model. In particular, the present invention can beimplemented locally at the health care provider organization or at adifferent site. In the latter case, the system implementation site andthe health care provider organization facility can be connected via aprivate network or through the Internet. The most significant advantagesof using the Internet relate to (1) the ability to maintain a singlelearning management or content database that can be kept updated in areliable manner so that knowledge granules may be accessed withoutconcern as to quality or whether it is accredited. and (2) the abilityto access content ubiquitously, i.e., without the need to be connectedto the health care organization provider network.

FIG. 2 shows a block diagram showing one implementation of a systemaccording to the present invention in a use case associated with the useof an organization's internal network to access the system via theInternet. In the diagram, the client devices 32 that will access thesystem 34 may be deployed within the health care provider organization36. The system 34 can be accessed occur through a desktop or laptopcomputer 38, a tablet or similar computing device 40, or through acellular phone 42 that is connected to the health care providerorganization's internal network. This can be accomplished through awired or wireless connection in the health care provider organization 36communications network. The health care provider organization's EHRsystem 44 may be connected to the same internal communications network.The client devices 32 do not need to be connected to the EHR system 44via a network switch 46 in order to access the system 34. However, ifthe health care provider organization's EHR system 44 is not connectedto the internal network and therefore cannot be accessed, the system 34will not be able to upload patient data that has been captured in thehealth care provider organization's EHR system 44 since the last time insystem uploaded data from the health care provider organization's EHRsystem 44. In order to connect to the system 34 via the Internet 48, thehealth care provider organization 36 may use an Internet router 50. TheInternet router 50 can connect to its counterpart Internet router 52 inthe site where the system 34 is implemented—“the system implementationsite” 54. In order to communicate with the system 34, the Internetrouter 52 at the system implementation site 54 may be connected througha network switch 56. This scenario reflects a block diagram for the casediscussed in FIG. 1, where client devices 32 can access the system 34patient information database 60 and the learning managementcomputational engine 62.

In an implementation of the system where the implementation site isaccessed via the Internet by the health care provider organization,there is the possibility of accessing the system from any deviceconnected to the Internet via any ISP. This allows a health careprovider to access the learning management computational engine at anytime to consume the knowledge granules that have been suggested by thesystem. This use case is described in the block diagram of FIG. 3, whichis a variation of the block diagram of FIG. 2. In this use case, thehealth care provider is in a location outside the health care providerorganization facility and is not connected to the latter's internalnetwork. Therefore, the health care provider is in a location whereInternet access is controlled by a third-party, which could be an ISP.In FIG. 3, the client devices 32 that will access the system 34 are in aphysical location outside the health care provider organization 36 andwhere there is no access to the latter's internal network. In this case,the user would connect the client device 32 to the Internet 48 throughan internal network where a network switch 66 may connect the clientdevice 32 to an Internet router 68. The Internet connection can becontrolled by a third party, such as an ISP 70. The health care providercan then connect to the system 34, through the Internet router 52 at thesystem implementation site 54 which needs to be connected through anetwork switch 56. This use case is different from the one described inFIG. 1 in that the health care provider is not accessing the system atthe point of care, but rather is a location outside the health careprovider organization.

FIG. 4 provides a work context diagram for providing continuing medicaleducation to a health care provider based on his or her actual clinicalperformance recorded in an EHR or similar clinical data system. In FIG.4, a health care provider 72 or other members of the care teams to whicha health care provider 72 is assigned can access a system applicationserver 74, which orchestrates the system components: the patientinformation database 76 and the learning management computational engine78. The system application server 74 may also controls communicationswith the health care provider organization's EHR database 80.Embodiments of the invention provides that there may be a designatedcontent administrator 82 who is responsible for uploading, modifying, ormanaging content (knowledge granules) in the learning managementcomputational engine 78 and providing the necessary metadata so that thesystem application server 74 can identify the knowledge granulealternatives associated with the performance gaps for the health careprovider 72.

The system application server 74 can be implemented so that it makesdecisions regarding the order in which it presents knowledge granulesalternatives based on the explicit or implicit preferences of the healthcare provider 72. The explicit preferences are those that the healthcare provider 72 has explicitly selected as part of his or her profilein the patient information database 76, while the explicit preferencesare those derived from the context in which the health care provider 72has consumed suggested knowledge granules in the past. Embodiments ofthe invention can further provide that the content administrator 82 maybe required to first make a login request 84 to the system applicationserver 74, to which the application server can respond 86. The contentadministrator 82 could be in practically any physical location whenmaking the login request 84 to the system application server 74. Thecontent administrator 82 can be at the health care provider organizationand connected to its internal network as in FIG. 2 or in any otherphysical location where a client device can connect to the Internet, asin FIG. 3. The content administrator 82 may be a third-party contentprovider, a representative of a medical credentialing organization, orany other person designated to manage the knowledge granules in thelearning management computational engine 78. The only requirement forthe content administrator 82 to make a login request 84 is that theclient device being used by the content administrator 82 is connected tothe system application server. In the scenarios and use cases in FIG. 2and FIG. 3, this implies having internet connectivity. If the loginresponse 86 is successful, the content administrator 82 can make arequest to add, modify or manage content 88 to the system applicationserver 74. In order to provide a response to the request 88, the systemapplication server 74 would connect to the learning managementcomputational engine 78 and relay the request 90. The learningmanagement computational engine may respond 92 to the system applicationserver's request 90 by confirming that the latter was completed. Thesystem application server may then respond 94 to the contentadministrator's request to add, modify, or manage content 88. Thecontent administrator's activities can ensure that the learningmanagement computational engine 78 has the required knowledge granulesto suggest content to the health care provider 72.

Embodiments of the invention can provide that the health care provider72, whether at the point of care or elsewhere, may be required to firstmake a login request 96 to the system application server 74, to whichthe application server can respond 98. The health care provider 72 maywant to add or modify data in the system related to his profile andcontent delivery preferences. This can be accomplished by submitting arequest to add or modify data 100 to the system application server 74,which may generate the appropriate request 102 to the patientinformation database 76. Once the request is completed, the patientinformation database 76 may respond 104 to the system application server74 request 102, which may be communicated back 106 to the health careprovider 72.

One aspect of the present invention is the evaluation of health careprovider performance against a set of thresholds or standards todetermine if there are any gaps and making continuing medical educationrecommendations specifically focused on developing the knowledge andskills needed to bridge the identified gaps. In FIG. 4, there is nospecific action that needs to be taken by the health care provider 72 toevaluate performance. This can be implemented in a number of ways,including a periodic evaluation of provider performance at a fixed timeinterval (e.g. monthly or quarterly), evaluating performance at aparticular point in the care plan stored in the patient informationdatabase 76 for each patient, or evaluating provider performance aftereach encounter at the point of care. In the latter case, rules can bedefined in the patient information database 76 to avoid notificationfatigue. In addition, the appropriate interface may be implementedbetween the system application server 74 and the health care providerorganization EHR database 80 to identify when a patient encounter hasoccurred.

Therefore, the system application server 74 may commence the health careprovider 72 performance review whenever it is triggered by theestablished business rules.

The first step is to ensure that the patient information database 76 hasupdated information with regards to the performance of the health careprovider 72. Thus, the system application server 74 may initiate arequest for patient data 108 associated with the health care provider 72to the health care provider organization EHR database 80, which may bean external system. If the request 108 is successful, the providerorganization's EHR database 80 may respond 110 with available patientdata. The system application server may then initiate a request 112 tothe patient information database to add or modify data associated with ahealth care provider 72 and his or her patients. If successful, thepatient information database 76 may respond 114 confirming that therequested addition or modification of data 112 was successful.

The system application server may request the necessary health careprovider 72 performance data from the patient information database 116.Once the system application server 74 receives the response 118 from thepatient information database 76, it may initiate a request 120 to thelearning management computational engine 78 to identify the availableknowledge granules that are associated with the specific health careprovider 72 performance gaps identified in the patient informationdatabase 76. The learning management computational engine 78 may respond122 with the links to the appropriate content.

As previously discussed, system application server 74 can be implementedso that it makes decisions regarding the order in which it presentsknowledge granules alternatives based on the explicit or implicitpreferences of the health care provider 72. The explicit preferences maybe those that the health care provider 72 has explicitly selected aspart of his profile in the patient information database 76, while theexplicit preferences may be those derived from the context in which thehealth care provider 72 has consumed suggested knowledge granules in thepast. The use of health care provider preferences would require that theinformation be included in the system application server request 116 tothe patient information database 76 as well as the request 120 to thelearning management computational engine 78 to obtain the implicitpreferences. The information on the appropriate content (knowledgegranules) may be presented to the health care provider 72 in the form ofan asynchronous alert 124.

FIG. 5 provides a work context diagram for the computational engine,which can absorb data associated with patient risks 125 and feed that tothe patient risk factor machine learning system 126. The patient riskfactor machine learning system may process the using risk detectionlogic and other processing algorithms. This data may flow down to thepatient workflow assignment system 127 where assignment logic and rulesmay be executed. The system 127 may also monitor for workflow taskscompletion. From here, the Continuous Education (CE) deployment system128 can absorb the information from the patient workflow assignmentsystem 127 to provide users with recommended CE based on the preferencesand previous performance, leveraging the CE library and promotingcompletion through the certifications and rewards and recognitionsreport. Each module may join with data from the train patient outcomereinforcement learning model 129 which may monitor results to identifyrequired training and identify improvements in processes.

FIG. 6 provides additional details on the functionality of the patientrisk factor machine learning system. This system may initiate a dataclean-up process 131 which may prepare the patient information data 132prior to the machine learning algorithm analyzing the data to predictpatient adverse health outcomes using social determinants and clinicaldata 133. The results of the algorithm may be processed 134 and passedto the risk factor score machine learning algorithm 135, which may alsoreceive new input data 136. This algorithm may identify if the predictedpatient outcomes are correct per the patient results data 137. If theprediction is incorrect, the data may be fed back to the machinelearning algorithm 133 to retrain the algorithm. If the prediction iscorrect, the results 140 may be used to identify related CE content tags138 and the results 140 may be fed into the patient assignment workflow139 in FIG. 7.

FIG. 7 provides additional details on the functionality of the patientworkflow assignment system. Leveraging the results from the patient riskfactor system 142, this system may train the machine learning algorithmon patient workflow assignments 143. This data may be processed 144, andthe algorithm may provide workflow recommendations 146, taking in inputdata on workflow actions 145. This system may be monitoring user actionsto confirm workflow application 147. If the workflow action was notexecuted, then it may tag the gap 148. If it was completed, then it mayanalyze the prediction for the patient outcomes 149. If the predictionis incorrect, the data may be fed back to the machine learning algorithm151 to retrain the algorithm. If the prediction is correct, the results150 may be stored and submitted to the CE deployment system in FIG. 8.

FIG. 8 provides a diagram of the CE deployment system. This system maytake in tagged CE content 157 from the patient risk factor machinelearning system and risk score tags 152, workflow variation tags 153from the patient workflow assignment system and user preference 154 andlearner profile tags 155 stored in the system itself. The matchingengine 156 may process this data and feed it to the machine learningalgorithm to determine recommended CE 158. This data may be processed159 and machine learning algorithm 159 may receive data from the system161 to determine user actions align with the recommendations 162. If itdetermines that the CE was not consumed, it may feed the data back tothe machine learning algorithm for retraining 158. If it was completed,then it may analyze the prediction for the patient outcomes 163. If theprediction is incorrect, the data may be fed back to the machinelearning algorithm 158 to retrain the algorithm. If the prediction iscorrect, the results 164 may be stored.

Thus, a method for providing education suggestions based on actualperformance has been disclosed. The present invention contemplatesnumerous variations in the type of information used, the topology of anetwork used to deliver the information, the type of devices used toaccess the information, the implementation model for the associatedsystem, and other variations within the spirit and scope of theinvention.

All the features disclosed in this specification, including anyaccompanying abstract and drawings, may be replaced by alternativefeatures serving the same, equivalent or similar purpose, unlessexpressly stated otherwise. Thus, unless expressly stated otherwise,each feature disclosed is one example only of a generic series ofequivalent or similar features.

Many alterations and modifications may be made by those having ordinaryskill in the art without departing from the spirit and scope of theinvention. Therefore, it must be understood that the illustratedembodiments have been set forth only for the purposes of examples andthat they should not be taken as limiting the invention as defined bythe following claims. For example, notwithstanding the fact that theelements of a claim are set forth below in a certain combination, itmust be expressly understood that the invention includes othercombinations of fewer, more or different ones of the disclosed elements.

The words used in this specification to describe the invention and itsvarious embodiments are to be understood not only in the sense of theircommonly defined meanings, but to include by special definition in thisspecification the generic structure, material or acts of which theyrepresent a single species.

The definitions of the words or elements of the following claims are,therefore, defined in this specification to not only include thecombination of elements which are literally set forth. In this sense itis therefore contemplated that an equivalent substitution of two or moreelements may be made for any one of the elements in the claims below orthat a single element may be substituted for two or more elements in aclaim. Although elements may be described above as acting in certaincombinations and even initially claimed as such, it is to be expresslyunderstood that one or more elements from a claimed combination can insome cases be excised from the combination and that the claimedcombination may be directed to a subcombination or variation of asubcombination.

Insubstantial changes from the claimed subject matter as viewed by aperson with ordinary skill in the art, now known or later devised, areexpressly contemplated as being equivalently within the scope of theclaims. Therefore, obvious substitutions now or later known to one withordinary skill in the art are defined to be within the scope of thedefined elements.

The claims are thus to be understood to include what is specificallyillustrated and described above, what is conceptually equivalent, whatcan be obviously substituted and also what incorporates the essentialidea of the invention.

What is claimed is:
 1. A method for providing continuing education to ahealth care professional comprising: continually assessing,electronically, via a system application server, actual clinicalperformance of the health care professional as recorded in a clinicaldata system; automatically creating a set of granules of health careknowledge in a database on the system application server, the set ofgranules of health care knowledge include information, skills, andaptitudes that are intended to improve performance of the health careprovider; associating continuing education credits with each granule ofhealth care knowledge of the set of granules of health care knowledge;analyzing the actual clinical performance of the healthcare providercontinually to determine whether the actual clinical performance fallswithin defined thresholds; training patient health risk factor machinelearning models to generate patient health risk factor scores usingsocial determinants and clinical data that correlate workflow assignmentto patient outcomes; training workflow assignment machine learningmodels to assign patients to workflows to achieve improved patientoutcomes; using the workflow assignment machine learning models with thepatient health risk factor scores to assign patients according to thepatient health risk factor score to a workflow to improve assignmentlogic and rules for achieving patient outcomes; training continuingeducation machine learning models with data from the patient risk factormachine learning models, with workflow variation tags from the patientworkflow assignment system, and with user preferences to identify whichof the set of granules of health care knowledge is correlated withimproved patient outcomes; defining a set of configurable conditions onthe system application server that use data from the clinical datasystem to trigger granule suggestions to the health care provider; usingthe continuing education machine learning models to assess knowledgegranules that are deployed in accordance with the learner's preferencesand learning styles; electronically delivering the health care knowledgegranule based on the established preferences of the health care providerautomatically as triggered by clinical performance; recording the healthcare provider's interaction with the health care knowledge granule;determining whether the interaction with the health care knowledgegranule is indicative that the health care provider consumed the healthcare knowledge granule; and recording continuing education credits basedon the delivered health care knowledge granule if there is indicationthat the health care provider consumed the health care knowledgegranule.
 2. The method of claim 1 wherein the clinical data system is anelectronic health record (EHR).
 3. The method of claim 1 wherein theeducation credits can be substituted for electronic badges or other formof reward and recognition.
 4. The method of claim 1 wherein theconfigurable conditions include data from a specific one of the healthcare providers and data from one or more care teams to which the healthcare provider is assigned.
 5. The method of claim 4 wherein theconfigurable conditions include comparisons of the health care provideror care team data against at least one of (1) established carestandards, (2) the performance of other care teams within a providerorganization, and (3) performance of other care teams that are outsidethe provider organization, independently of their performance againstestablished care standards.
 6. A computer-assisted method for deliveringhealth care knowledge to a health care provider, based on actualclinical performance recorded in a clinical data system, comprising:participating in interactions with patients and recording associateddata in the clinical data system in communication with a systemapplication server; automatically triggering one or more suggestedhealth care knowledge granules based on performance of the health careprovider, the one or more suggested health care knowledge granuleselectronically stored in a database of the system application server;obtaining a request, by the system application server, from the healthcare provider to deliver the suggested health care knowledge granules;delivering, by the database of the system application server, thesuggested granules that are triggered for and/or requested by the healthcare provider; recording an interaction of the health care provider withthe health care knowledge granule on the system application server;determining, by the system application server, whether the interactionbetween the health care provider and health care knowledge granule wassubstantive; training workflow assignment machine learning models toassign patients to workflows to achieve improved patient outcomes;training patient health risk factor machine learning models to generatepatient health risk factor scores using social determinants and clinicaldata that correlate workflow assignment to patient outcomes; trainingcontinuing education machine learning models with data from the patientrisk factor machine learning models, with workflow variation tags fromthe patient workflow assignment models, and with user preferences toidentify which knowledge granules are correlated with improved patientoutcomes and under what conditions; using the continuing educationmachine learning models to assess knowledge granules that are related tothe learner's preferences and learning styles; using the workflowassignment machine learning models to assign patients according to thepatient health risk factor score to a workflow to improve assignmentlogic and rules for achieving patient outcomes; and recording, on thesystem application server, continuing education credits based on thedelivered health care knowledge granule if there is indication that thehealth care provider had a substantive interaction with the health careknowledge granule.
 7. The method of claim 6 wherein the clinical datasystem is an electronic health record (EHR).
 8. The method of claim 6,further comprising triggering one or more health care knowledge granulesuggestions based on performance of the health care provider and one ormore care teams to which the health care provider is assigned.
 9. Themethod of claim 6, further comprising delivering a requested health careknowledge granule based on saved or inferred preferences of the healthcare provider using machine learning algorithms.
 10. The method of claim6 further comprising: accessing at least one related granule of healthcare knowledge; and documenting continuing education credits based onthe at least one related granule of health care knowledge.
 11. Themethod of claim 6 further comprising extracting clinical data from theclinical data system used by the health care provider to record apatient's demographic and clinical information, including at least oneof problem lists, clinical procedures, diagnostic codes, ancillaryservice orders, and results of the ancillary service orders.
 12. Themethod of claim 6 further comprising computing at least one of qualityclinical measures, gaps in care, outcomes, and other objective measuresfor the health care provider.
 13. The method of claim 6 furthercomprising comparing at least one of the health care provider's qualityclinical measures, gaps in care, outcomes, and other objective measuresagainst at least one of (1) established clinical care standards, (2)performance of other care teams within a provider organization, and (3)performance of other care teams that are outside the providerorganization, independently of their performance against standards usingreinforcement learning algorithms focused on improving patient outcomes.14. The method of claim 6 further comprising triggering health careknowledge suggestions based on the results of at least one of acomparison of quality clinical measures, gaps in care, outcomes, andother objective measures for the health care provider.
 15. The method ofclaim 14 further comprising saving the health care knowledge suggestionsas alerts in a queue that the health care provider can respond towhenever it is convenient.
 16. The method of claim 15 further comprisingcapturing a response to the alerts by the health care provider.
 17. Themethod of claim 6 further comprising delivering the suggested healthcare knowledge granules that were approved for delivery by the healthcare provider in a manner, format, or method associated with saved orinferred preferences of the health care provider.
 18. The method ofclaim 6 further comprising recording details of an interaction of thehealth care provider with the delivered health care knowledge granule.19. The method of claim 18 further comprising using details of thehealth care provider's interaction to determine whether the health careprovider's interaction with the health care knowledge granule wassubstantive or not.
 20. The method of claim 19 further comprisingissuing the continuing education credits in favor of the health careprovider for any knowledge granule where the health care provider'sinteraction was substantive.